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DESCRIPTION OF THE PROJECT 



Orange County Public Schools serves approximately 90 , 000 
students in greater Orlando area. Approximately 8% of the 
total, school population is Hispanic. The district provides 
bilingual services K-8 and ESOL services K-12 to students 
who are limited in English proficiency {LEP) . Services to 
handicapped and gifted students are provided through the 
Department of Student Services and Exceptional Education. 

Bilingual exceptional education in Orange County , Florida 
began in 1985 in response to concern for a growing 
population of exceptional students who were also limited in 
English proficiency. The purpose for initiating a federally 
funded Title VII project was to expand existing district 
services to the handicapped LEP students. 

The grant proposal stated that there were two major goals 
for improving and expanding the district's exceptional 
transitional bilingual education program: 

1. to develop , demonstrate and package methods, procedures 
and materials for identifying, placing and instructing 
handicapped LEP students 

2. to develop, demonstrate and pacrage personnel and parent 
training components that the district and other school 
systems can use on a continuing basis. 



Currently, the project is staffed by a project manager, a 
full-time resource specialist, and a part-time parent 
liason/social worker. Six teachers and four full-time aides 
are employed at two elementary schools, one middle school, 
and one high school. The project currently serves 74 EMH 
and LD students who are LEP, and as Table 1 indicates, the 
numbers are growing; the students are primarily from Puerto 
Rico. 
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Table 1 Breakdown of Handicapped LEP Students in Orange 
County Program 



Level 


Year 1 


Year 


2 


Year 


3 




LD 


EMR 


LD 


EMR 


LD 


EMR 


K-5 


22 


15 


25 


24 


23 


20 


Middle School 


9 


3 


13 


4 


17 


6 


High School 


* 


* 


5 


1 


6 


2 


Total 


31 


18 


43 


29 


46 


28 



* As part of the project's expansion activities , the high 
school program was initiated during the second year of the 
project* 



The specific objectives of the project were to: 

1. Expand the existing elementary program for handicapped 
LEP students. 

2. Develop exit criteria for moving students from the 
exceptional transitional bilingual education program. 

3. Develop and pilot test a secondary program for 
handicapped LEP students. 

4. Provide inservice training of teachers and support 
personnel in strategies and techniques for working 
with handicapped LEP students. 

5. Develop and implement a special education component to 
train parents of handicapped LEP students in the goals 
and objectives of the district's program, the implica- 
tions of their children's handicaps, their role in 
planning their children's educational future, and in 
strategies, methods, and materials to reinforce their 
children's schooling in the home setting. 

This paper will report selected results of achievement of 
project goals and objectives which specifically relate to 
feedback provided by the external evaluator. 



EVALUATION DESIGN 

The design of this evaluation consisted of a process 
evaluation to assess the extent to which the project has 
achieved its objectives during the first two years and a 
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product evaluation to assess student performance across the 
three year period. Data were collected by means of 
interviews with project personnel , site visits to the 
project schools , review of documentation of implementation 
and review of pupil progress data. 

Initial contact as an evaluator with Orange County Schools 1 
bilingual exceptional education project was after the 
project had been operating for about two months. At that 
time the project director, resource specialist, and I met to 
discuss my role as external evaluator of the project. We 
also established the methods to be used for evaluating the 
project and the types of data collection required for each 
of the three years of the project. We set up timelines for 
receipt of data and for my visits to the district. 

During the first year, my input was somewhat supervisory in 
nature, particularly with regard to the data collection and 
maintenance procedures. The following data were decided 
upon during the first year and are being kept over the three 
years of the project to assess pupil performance and 
progress: 

Name 
ID 

Date of Birth 

Attendance (number of days present over number of days in district) 
Exceptionality (EMH, SLD resource, SLD Self-contained, Etc.) 
Date entered into program 

Teacher rating of L1/L2 proficiency (natural language ratings) 
Checklist objectives L1/L2 (see Attachment 5) 
Brigance L1/L2 scores: 

Word Recognition grade 1 1 placement; 

Reads Orally level; 

Reading Comprehension level; 

Math grade level in 4 basic computations LI; 

Math word problems LI 
Language Assessment Battery L1/L2 scores: 

level administered; 

speaking, listening, reading, and writing subtest raw scores; 

total battery raw scores; 
Exit dates, reasons for exit, school, and ESE placement 
Exit Follow-up results: teacher survey data, attendance, Checklist 
results 

Any drop-outs from school 
Any retainees 

Unfortunately, my suggestions for utilizing a microcomputer-based 
system for maintaining the project data were not implemented due to 
time and budget constraints. However, the project* s resource 
specialist developed a one-page summary data sheet for each project 
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student (see Attachment 1) on which these data are entered and then 
forwarded to the evaluator for review and subsequent analysis. 



IMPLEMENTATION 

My objective for the second year's evaluation was to get 
some measure of implementation of the project's goals. The 
project director and I developed a checklist of items based 
on the things she and the teachers wanted to see 
accomplished during the three years of the project (see 
Attachment 2) . Our goal was to measure constructs unique to 
bilingual special education and different from regular 
special education, although there is some overlap. Key 
features of the program are typed in bold face. Each of the 
items on the checklist were rated during three observations 
at the beginning , middle , and end of the second year of the 
project. Results of these observations were communicated to 
the project director who in turn communicated the results to 
individual teachers; general observations about the site 
visits were also discussed in group meetings. 

On the whole , the project teachers were implementing key 
elements of the program quite effectively (see Table 2 and 
Figures 1-8). For purposes of analysis , I combined the key 
elements into their larger descriptive categories for all 
six project teachers. 



Table 2 Mean Ratings of Key Elements of Implementation 
Checklist Across Time (n=6) 

Time 1 Time 2 Time 3 Overall 



Methods 


2.99 


3.69 


3.94 


3.56 


Content 


3.06 


3.58 


3.81 


3.51 


Materials 


2.67 


3.75 


3.25 


3.27 


Environment 


3.00 


3.70 


3.83 


3.53 


Evaluation 


3.30 


3.50 


3.92 


3.60 


Management 


3.23 


4.03 


3.75 


3.68 


Teacher 


4.25 


4.60 


4.45 


4.45 



4 

6 



Examination of the ratings on the checklist indicate that 
the project teachers were increasingly implementing the 
projects goals; their greatest strength, according to the 
checklist was in the area of teacher characteristics, which 
includes teacher warmth, high expectations, specific praise, 
and humor. Upward trend lines were also observed in the 
areas of Method, Content, Environment, and Evaluation. 

Feedback using the checklist also resulted in several 
important changes during the implementation phase of the 
project: 

1. Increased time on task. After the initial site visit 
using the checklist, I recommended that each of the teachers 
be reminded of the need to keep the students on-task for 
efficient use of classroom time. Subsequently, the project 
teachers discussed time-on-task issues at one of their 
monthly meetings so that the next visit revealed significant 
improvement in this area. In the case of the resource LD 
teacher at the elementary school site, a half-time LD 
teacher was added to work with the non-LEP LD students so 
that the bilingual LD teacher would have more time to 
interact with and monitor the LEP students. 

I also suggested that time might be more effectively 
utilized if each student had his or her own folder to pick 
up at the beginning of class; this folder would contain 
instructions for daily work based on objectives from the IEP 
and a list of materials, dittos, resources, etc. to be used 
by the student. In this way, students would not have to 
spend time waiting for the teacher to tell them what to do 
or to check his or her work, but could go to the designated 
item in the folder. Work could then be checked at the 
teacher's convenience. Several of the teachers have chosen 
to organize their instruction in this way, particularly for 
the resource students. 

2. Use of Teacher Aides Initial site visits to the project 
classrooms revealed the project teachers 1 heavy reliance on 
their teacher aides to conduct one-on-one instruction with 
students having difficulty or who were especially slow. 
Rather than serving the purpose of individualizing 
instruction, this practice served to isolate these students 
from the teacher, the other students, and the group lessons 
being taught. It appeared that the students having the 
greatest needs were not receiving an appropriate amount of 
interaction with the person most qualified to assist them, 
the classroom teacher. After simply pointing this practice 
out to the project staff, their awareness of this behavior 
and its potential effect rose, and the use of aides began to 
change. I suggested that the aides might be more 
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appropriately used to monitor group sea\- work after a group 
lesson while the teacher worked with an individual or small 
group needing special attention or instruction. Subsequent 
visits to the project classrooms indicate a more balanced 
use of teacher aides by most project teachers. 

3. Greater use of the second language Although the program 
at OCPS is a transitional bilingual program, my first visit 
to the school sites revealed a strong emphasis on the home 
language (Spanish) , with entire observation periods going by 
without any English being spoken by the teacher, although 
many of the students were speaking English among themselves. 
My suggestions for striking a better balance of the two 
languages were: a) to build English language skills through 
greater usage (especially where concrete referents v;ere 
involved; e.g. "Line up please."} , b) to emphasise ESOL, 
and c) to utilize the home language to clarify subject area 
content when necessary and expedient. This advice was 
communicated to the project staff , and after a great deal of 
discussion by everyone concerned , was initiated and 
implemented. Consultants were utilized to inservice the 
project staff (as well as regular classroom and special 
education teachers) in ESOL methodologies and techniques and 
in balancing the two languages for optimal instructional 
effectiveness. Greater use of English by both teachers and 
students was observed during the evaluator' s following site 
visits using the checklist. 

The implementation checklist proved to be a useful tool for 
observation. In the first place, I believe that it was 
helpful for the teachers to be reminded once again of the 
key elements toward which they were to direct their 
energies. Secondly, the feedback from an "objective 
observer" was viewed as necessary and worthwhile. The 
checklist also enabled me to focus my observations and look 
at progress over time. 

EXIT CRIT3RIA 

In addition to the site visits using the implementation 
checklist, the evaluator worked with the project director 
and resource specialist on establishing exit criteria for 
handicapped LEP students; these criteria were later 
published in the districts procedures manual. A number of 
criteria were utilized based on evidence from the literature 
and suggestions from practicioners in the fields of 
bilingual and special education. An excerpt from the 
procedures manual is included in Attachment 3. 

At the beginning of the second year of the project, data 
were collected by the external evaluator to provide 
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information on the English oral language proficiency scores 
of regular (non-LEP) EMH and SLD students in the district, 
as measured by the Language Assessment Battery* The purpose 
of the testing was to establish a baseline proficiency level 
in listening, reading, and writing for regular special 
education and provide a research-based model fori 
establishing exit criteria from bilingual special education 
to regular special education, A stratified random sample of 
158 native English-speaking students was obtained in second 
fourth, and sixth grades. These students were tested using 
the Language Assessment Battery, an instrument for measuring 
both oral and academic language proficiency in English or 
Spanish. This particular instrument is used by the district 
for testing all students whose native language is Spanish, 
and was utilized in the project as a standardized measure of 
progress over time. Preliminary results of total LAB scores 
of handicapped English-speaking second, fourth, and sixth 
grade students are presented in Table 3. 



Table 3 Mean Percentile Ranks for Total LAB Scores of 
English-speaking Exceptional Students 







Level I 


Level II 


Level III 






(grade 2) 


(grade 4) 


(grade 6) 


EMH 




13 


8 


11 






(n=8) 


(n=10) 


(n=8) 


SLD 


Resource 


60 


35 


22 






(n=12) 


(11=19) 


(n=12) 


SLD 


Full-time 


24 


20 


22 






(n=14) 


(n=10) 


(n=13) 



Preliminary results of the LAB testing indicate that the 
current criteria of a percentile rank of 30th percentile or 
better for exit may be high for EMH students at grades 2, 4, 
and 6 and that the criteria may be slightly high for 
full-time SLD students in the three grades tested, as well 
as for SLD resource students in grade six. It appears that 
the Level I test was easy for the SLD resource students in 
grades two and four. These data will be correlated with the 
actual spring LAB scores of successfully exited project 
students at the end of the third year of the project. 
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PUPIL PROGRESS 



As indicated earlier in the paper , a large amount of data is 
being compiled and maintained on the project students. 
These data are kept for both current and exited project 
students. Unfortunately , a control/comparison group design 
is not feasible due to the small number of comparison 
students at each grade level who are handicapped and LEP, 
but who are not receiving services from the project. 
Evaluation of pupil progress, therefore, will be conducted 
as a time series design , where student progress will be 
plotted across the three years of the project on the LAB, 
Brigance, and L1/L2 Checklist. Results will be presented 
according to exceptionality and grade levels (primary, 
intermediate, middle, and high school) . 

The project in Orange County saw the exit of a number of its 
project students to regular special education classrooms 
during the three years of operation, so it was important to 
follow these students carefully in order to complete the 
data for evaluation. Based on my recommendation, all exited 
project students are given the same battery of tests by the 
resource specialist as the project students. In addition, a 
follow-up form was developed by the resource specialist (see 
Attachment 4) to be sent to the classroom teacher each 
semester after exit to ascertain if there were any 
difficulties in adjustment or need for additional 
assistance. This form is returned to the project office 
along with a copy of the student's report card and reviewed. 
Any difficulties are promptly addressed by the project 
staff. In nearly all cases, the exited students have done 
remarkably well in regular special education classes. 



SUMMARY 

Part of the evaluator' s job is to establish himself or 
herself as a team player on any important project. The 
interaction of OCPS' Title VII project staff and the 
external evaluator has been one of collaboration since the 
very beginning of the project. The project administration 
and staff have been very open to suggestions for improvement 
throughout the course of this project. The t3am approach 
has made for effective utilization of feedback from fhe 
project evaluator for ultimate improvement of services to 
handicapped LEP students in Orange County. 
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ATTACHMENT 1 



DATA SHEET 



NAME: 



ID I : 



D.O.B. 



/ / 



NAME: 



ID 



1st LANGUAGE: 



SCHL YEAR 
88 - 89 






FHIRY PATE .. 
ESE ESEB 


STAFFING 


EPC 


EXCEPTIONALITY UES) 


GRADE: 














S/E w Spanish 
English 



EXIT DATE: 



EXIT PLACEMENT: 



, EXIT REASON: 
COMMITTEE REC: PARENT REC: MOVED*. 



DROPPED OUT: 
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j SCHOOL YEARS 05-86 TO 88-89 




CATtViUKi 


0S-C6 


06-87 


87-86 


88-89 


1 nDTPRKlPT 


■ 




nmf d fading 


DATE SCORE 


DATE SCORE 


DATE SCORE 


DATE 


SCORE 


SPANISH 


















ENGLISH 


















1 READING COHP 
SPANISH 


















ENGLISH 


















1 LISTENING COKP 
SPANISH 


















ENGLISH 


















[ COMPUTATION I 
ADDITION 


















SUBTRCTN | 


















MULTIPLI 


















DIVISION 


















WORD PROBLEMS 
SPANISH 


















ENGLISH 


















READINESS 
SPANISH 


















ENGLISH 


















RETAINED 


YES 


NO 


| YES 


NO 


YES 




YES 


NO 



















.BESLCOPJLAVAILABLL 



02- 



ATTACHMENTS 



Schoci: 



ESEB Program Checklist 
Teacher/Class: 



ATTACHMENT 2 



Date: 



Key instructions? Matures 
1987-88 in be type 



Fully 
implemented 



4 



Not 
implemented 
at this time 
1 



Management of Instruction : 



Methods/Techniques- 



cooperative group activities 



use of poetry, rhymes, songs 



inclusion of students' culture 



multisensory methods 



individual remediation 



meaningful language 



peer tutoring 



recognition of learning styles 



narratives 



content-based ESOL 



Content 
language development (Eng.) 
language development (Sp.) 
reading (Sp.) 

concept dev. - science (Sp.) 
concept dev. - math (Sp.) 
concept dev. - social studies (Sp.) 
content approp. to skill levels (IEP) 
critical thinking skills 
learning strategies 
"vocational" skills 



Materials 
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ESEB Program Checklist 
School: Teacher/Class: Date: 



\C&\t instructional fpa turps 

1987-88 in bold type 


Fully 
5 


4 


3 


2 


Not 

IlllUldllCI llt?U 

at this time 
1 






































Environment 












opportunity for practice 












friendly interactions 




































• 












Evaluation/Assessment 












monitoring of student work 












language sampling. 












precision teaching 






















• 






































Management of Behavior: 












on-task behavior 












behavior management system 












use of teacher aide 












• 






















• 














Teacher Characteristics: 












warmth 












high expectations 












speciTic praise 












humor. 
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POLICY ON PRnumtK^ sxc ~ TTONA , ^ 
Public Law cua 

ac h c e o m n p e rf a :°r ^-^Si^nS^ur--"* to 

accompliahad xn a language the studio* . *? sar Vice 3 must be 
student w.ho is limited English VJH* * u " der 3*^ds. If a 
.exceptional education program " ST X f XBnt ia st ^d into an 
have determined that the Kir 9 3ta ^i"g commitee must 
dominant language, .nd^tX^n^tVr ^ StUd *""'* 

populations. UalJl- onl y the "following l£P 

and to the ESOL or biHnn,,", * other handicaps, 

r 

ELIGIBILITY (Di rect Services > 

A* On an individual evaluation i 

student is determined to be llVll/T^ th » 
us.ng either of >th „ stowing •^ 1 t r 1 'J° t r s f 11,n 9 U al service's 

student mu " score at or ii"J 9U *ii ES0L offlC9 - Th * 

score at or below the 20Xite in English. 

The student auet^re'st'Teve, ^i"?**^-*- 
the s°? u d l . n n d t iV i. d . U t*, 1 ::?r l . U n*c t .% n r , 1 t% d r"'i 1 a nl ?o t ' r " d SP * ni3h ' ' 



ELIGIBrtLgr ycconauIt^tive Services) 

is dete^r^dV^^r^^^f^ r° f i5i enc ^ the 

The 



1. The Language Assessment Batterv (LAB) Th*«= • ■_ 
administered by the Bilingual/St Offfci Sne 
must scare at the 30'/.tile or below in English 
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2. The Language Assessment Scales (LrtS) This will be 
administered by a bilingual school psychologist. The 
student must score at Level III or below. 

B. On individual evaluations administered by a bilingual 
school psychologist in the student's native language through 
an interpreter* the student meets entrance criteria for an 
exceptional education program as specified by state and 
district criteria <refer to other sections of this handbook). 



REFERRAL » EVALUATION AND STAFFING 

Evaluation of English and native language competancies will 
involve the f ol lowing procedures and personnel : 

A. Screening of language dominance or English proficiency by 
the Bilingual /ESOL Office to determine eligibility for 
bilingual and/or ESOL services, 

B. Notification to the home or receiving school by the 
Bilingual /ESOL Office if the result* of the screening 
indicate the need for further assessment. (This will be 
noted on the Student Primary /Home Language Survey form.) 

C. Referral by the home or receiving school to Psychological 
Services. Note s As with any other referral? an effort 
should be made by the school* by way of Educational Planning 
Team meetings or other methods 9 to address the needs of the 
student before referring the student for psychological 
testing. With LEP students 9 the contribution of cultural 
factors to a student's learning or behavior patterns should 
be kept in mind. 

D. Evaluation by a bilingual school psychologist in English 
and in the student's native or dominant language to provide 
additional information regarding levels of proficiency and 
possible handicapping conditions. 

E. , Completion of the Language Proficiency Checklist (which 
may be obtained by calling the Curriculum Resource Teacher 
for the ESE Bilingual Program) by bilingual or other 
personnel who have been working directly with- the student. 

F. Completion of the Speech/Language Checklist by bilingual 
or other personnel who have been working directly with the 
student, and referral to the bilingual speech/language 
pathologist if preliminary screening or individual 
evaluation results indicate a need. 

For a list of required -forms and persons required to be at 
the eligibility and placement staffing, refer to other 
sections of this handbook. 
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Additional forma required for eligibility staffing* of LEP 
students* 

Student Primary Horn© Language Survey Form 
Language Proficiency Checklist 
LEP IEP second page 

Additional required persons for eligibility staffing© of LEP 
students: 

Bilingual School Psychologist 

Curriculum Resource Teacher? ESE Bilingual Program (if 
the bilingual psychologist cannot be present) 



RE-EVALUATION AND DISMISSAL 

When the teacher or parent feel the student should be 
considered for placement in a "regular 1 * exceptional 
education class* i.e. 9 one which does not include services 
from the ESE Bilingual Program* a re-evaluation must be 
completed. 

For required forms and procedures for re-evaluations* refer 
to other sections of this handbook. 

Additional required forms for re-evaluation EPCs of LEP 
studentss 

Language Proficiency Checklist 

Results of updated LAB or LAS (within 6 months) 

Additional required persons for re-evaluation EPCs of LEP 
students: 

Bilingual School Psychologist 

Curriculum Resource Teacher for ESE Bilingual Program 
(if bilingual psychologist cannot be present) 

Guideline* -for determining when a student should be 
considered for placement in a "regular" exceptional edu cation 
class arer ae follower " — — 

A. Gradual mainstreaming into a "regular" ESE class has 
been successful, or gradual use of English by the ESEB 
teacher has been successful , as documented by progress 
reports and/or observations by teachers^ 

"B'. Progress is evident on the LEP IEP page, and the 
Language Proficiency Checklist shows mastery of all BICS 
(Basic Interpersonal Communication Skills), and a portion 'of 
CALP (Cognitive/ Academic Language Proficiency) skills. 

C. The teacher rates the student as being in the 
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Intermediate Fluency stage in English? according to the 
hierarchy of second language acquisition ( as described by 
in vhe Natural Approach) 

D. The student scores c.bove the 30%ile in English on the LAB 
or at Level IV or above on the LAS. 

Guidelines for determining when an LEP studen t should be 
dismissed from exceptional education are as follows: 

A. If the student? upon re-evaluation? demonstrates that 
he/she no longer has a need for exceptional education 
services? but still qualifies for bilingual or ESOL services? 
the student should be dismissed and referred to the 
Bilingual /ESOL Office. 

B. If the student? upon re-evaluation? demonstrates that 
he/she no longer has a need for exceptional education 
services and does not qualify for bilingual or ESOL services? 
the student should be dismissed in the same manner as any 
other ESE student. 

For required forms and procedures for dismissals? refer to 
other sections of this handbook. 
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ATTACHMENT 4 



December 1, 1987 



MEMORANDUM 



TO: 



SELECTED ESE TEACHERS 



FROM: 



JULIE LI VESA y9[^ U 



Senior Admin isirator 
Exceptional Education Program Development 



SUBJECT: EXITED ESE BILINGUAL PROGRAM STUDENTS 



In 1986, Orange County received a Federal Grant under Title VII which allowed the 
ESE section to expand the newly created Exceptional Student Education Bilingual 
(ESEB) Program. 

During the 1986-87 school year, several students who were in the program were 
exitedto regular ESE classes. In order to comply with the requirements of the grant, 
these students must have their progress monitored and continue to participate in 
the follow-up studies at least until June 1989. Attached to this letter is a form to be 
completed regarding these students. As the school year is now well under way, and 
students and teachers have had time to adjust and become familar with each other, 
it should not be difficult to do. 

Please take the time to complete the form as soon as possible. Return it with a copy 
of the student's report card via courier mail to the address indicated below. If you 
have any questions regarding the form, please call 423-9245* tor assistance. 

Please return all information requested by December 16, 1987 to: 

Maria Bri.ganti 

ESE Bilingual Resource Specialist 
Administrative Center 

3mr102 

cc: Maria Briganti 
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ESEB Program: CONSULTATIVE SERVICES 

Date: 

Student: 

D.O.B. Age: 

School: 1 : , 

Program: . 

Teacher & Program Subject Student has adjusted 

YES NO 



If the student has not adjusted? please describe any problem 
the student may be experiencing as you see it. 1 



If any adjustments have been made in curriculum ? techniques 
or methods please describe briefly. 



Would you be interested in receiving inservice training to 
assist you in working with Limited English Proficient 
exceptional students ? YES ND 



For further information contact the Orange County Public 
Schools ESEB office at 423-9849. 
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